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Overview  of  Existing  Global  Models 
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Unified  Dynamics 

•  All  limited-area  models  are 
nonhydrostatic. 

Resolutions  of  global 
models  are  approaching  the 
nonhydrostatic  limit  (~10 
km). 

•  Both  limited-area  and 
global  models  utilize  the 
same  equations. 

•  Engineer  a  common 
dynamical  core  for  both 
models,  then  change  grids, 
force,  and  boundary 
conditions. 


•  Unified  Numerics 

•  CG  is  more  efficient  for 
smooth  problems  at  low 
processor  counts. 

•  DG  is  more  accurate  for 
problems  with  sharp 
gradients  and  more 
efficient  at  high 
processor  counts. 

•  Both  EBGs  utilize  a 
common  mathematical 
arsenal. 

•  NUMA  allows  the  user  to 
choose  either  CG  or  DG 
for  the  problem  at  hand. 


Unified  Code 

Code  is  modular , 
with  a  common  set 
of  data  structures. 

New  time- 
integrators,  grids, 
basis  functions, 
physics,  etc.  may 
be  swapped  in  and 
out  with  ease. 

Code  is  portable: 
Successfully 
installed  on  Apple, 
Sun,  Linux,  and 
IBM. 


Non-Conservative  Form  (2NC) 


•Consider  the  compressible  Euler  in  Cartesian  coordinates  (not  spherical). 
•Mass  is  conserved  and  energy  can  be  conserved  (up  to  time-truncation) 
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•The  same  model  may  be  used  for:  global,  limited-area,  and  urban-scale  modeling 
(requires  different  grids,  initial  conditions,  boundary  conditions,  and  physics). 


Example  of  3D  Grids 


Mesoscale  Modeling  Mode  Global  Modeling  Mode  Global  Modeling  Mode 

(Cubed-Sphere)  (Icosahedral) 


NUMA  runs  in  either  Mesoscale  (Limited- Area)  or 
Global  Mode. 

Currently,  any  (hexahedra-based)  grid  can  be  used 
including  completely  unstructured  grids. 

Parallel  Domain  Decomposition  handled  by  METIS. 


Domain  Decomposition  via  METIS 


2:  Decomposition  of  a  “cubed 

1 :  Decomposition  of  a  3D  sphere”  using  96  spectral  elements 

Cartesian  domain  using  64  (fourth  order), 
spectral  elements  (fourth  order). 


3 :  Decomposition  of  an 
“icosahedral  sphere”  using  96 
elements  (fourth  order). 
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Rising  Thermal  Bubble:  3D 
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Orographic  Flows:  Linear  Hydrostatic  and 
Nonhydrostatic  Mountain  (3D) 

•Flow  ofU=20  m/s  in  an  isothermal  atmosphere. 

•LH  Mountain:  Solid  of  revolution  of  Witch  of  Agnesi:  Mountain  height  =  1  m 
with  radius  10  km. 

•Radiating  boundary  condition  implemented  on  lateral  and  top  boundaries  using 
an  absorbing  sponge  (Rayleigh  friction). 
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Linear  Hydrostatic  Mountain  (3D) 
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3D  Linear  Hydrostatic  Mountain  (Verification) 


Mountain  case  tests  orography  and  sponge  BC. 

Analytical  approximations  for  flow  over  a  LHM  available  in  Smith  (1980) 
on  mountain  surface. 

Decent  agreement  between  theory  and  numerical  model  for  downstream 
and  cross-stream  velocity  perturbations. 


Results  (Global):  Pressure  Wave 


•  Proposed  by  Tomita  and  Satoh 
(2004)  to  test  nonhydrostatic 
global  atmospheric  models. 

•  A  pressure  perturbation  is 
applied  to  a  hydrostatic, 
isothermal  atmosphere 


•Top  Level  zT=10  km 

•Cubed  Sphere  Grid  with  6x  12x  12x6  =  51 84 
elements  using  4th  order  polynomials  (~2 1 0  km 
horz.  Resolution  and  0.5  km  vertical  resolution) 

•Model  run  using  Semi-Implicit  TI  w/  5  s  time- 
step 
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Results  (Global):  Inertia-Gravity  Wave 


Potential  Temperature  after  48  hours 
with  (240x0.5)  km  resolution 


Urban-Scale  Modeling 


CPU  time  (s) 


CG:  Scalability  (Explicit  +  Semi-Implicit) 


Moderate  Processor  Counts:  110592  Large  Processor  Counts* 

elements  262144  elements 

Time- Steps 

Explicit  RK35:  At  =  0.001  s 
SI  BDF2:  At  =  0.01  s 


Summary  and  Outlook 


NUMA  Dynamical  Core  is  quite  mature: 

-  3D  and  MPI . 

-  Can  use  either  CG  or  DG  methods. 

We  will  continue  testing  the  dry  dynamics  and  improve  the  model  to  make  it 
as  user-friendly  as  possible. 

Future  Projects: 

-  We  are  implementing  sub-grid  scale  parameterizations  to  NUMA 

-  Need  more  resources  added  to  implement  sub-grid  scale 
parameterization,  data  assimilation,  testing,  etc. 
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Conservative  Form  (2C) 


►Mass  is  conserved  but  not  energy. 

►Conservation  (or  flux)  form  is  required  by  DG  machinery. 
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2D  Inertia-Gravity  Wave 

•Standard  test  suite  for  2D  mesoscale  problems  (Skamarock,  Doyle,  et.  al.) 

•Potential  temperature  perturbation  after  3000  s  for  250  m  resolution  (120 
by  1  by  4  elements)  and  10-th  order  polynomials. 


CG-NUMA 
run  with  1 
element  in  y- 
direction. 


2D  Density 
Current 
results  for 
Giraldo  and 
Restelli 
(2008). 


Results  (Global):  Pressure  Wave 


Acoustic  (perturbation)  field  shown  at  central  vertical  level  (5  km). 

Wave  returns  to  initial  state  after  about  32  hours,  yielding  a  speed  of  sound  of  348  m/s. 


Non-hydrostatic  Adaptivity  Examples 

(Muller,  Behrens,  Giraldo,  Wirth  2010) 
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Warm  air  bubble 
after  0  seconds. 

amplitude:  0.5  K 
order  of  polynomials:  3 
timestep  length:  5ms 
shortest  edge  length:  11m 
diffusion  parameter:  0.1m2/s 
time  integration  method:  RK35 
color:  pot.  temperature  -  300K 


(c)  August  2009,  A.  Mueller  (Mainz,  Germany) 


Rising  Thermal  Bubbles 


Two  (Warm/Cold)  Thermal  Bubbles 
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Overview  of  Existing  Limited- Area  Models 
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MPI  Dry  core 
under 

development. 


MPI  Dry  core 
developed. 
Physics  will 
be  added  this 
year. 
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Performance  of  the  NUMA  Model 
(3D  Rising  Thermal  Bubble) 


DG-NUMA  on  TACC  Ranger 
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Discontinuous  Galerkin 


16  Million  Grid  Points 


